Apparatus and method for light control in an in-vivo imaging device

ABSTRACT

A device and method for example operating an in vivo imaging device wherein the illumination is operated at a certain rate or range of rates, and images are transmitted from the device.

RELATED APPLICATION DATA

The present application is a continuation of U.S. application Ser. No.10/551,053, entitled “Apparatus And Method For Light Control In AnIn-Vivo Imaging Device” filed on Sep. 23, 2005, which is a nationalphase of International App. PCT/IL2004/000265, filed on Mar. 23, 2004,which in turn claims priority from Israel patent application 155046,filed on Mar. 23, 2003; in addition U.S. application Ser. No. 10/551,053is a continuation-in-part of U.S. patent application Ser. No.10/202,608, filed Jul. 25, 2002, which in turn claims priority from U.S.Provisional Application 60/307,603, filed Jul. 26, 2001, all of whichare incorporated by reference in their entirety.

BACKGROUND OF THE INVENTION

Devices and methods for performing in-vivo imaging of passages orcavities within a body are known in the art. Such devices may include,inter alia, various endoscopic imaging systems and devices forperforming imaging in various internal body cavities.

Reference is now made to FIG. 1 which is a schematic diagramillustrating an embodiment of an autonomous in-vivo imaging device. Thedevice 10A typically includes an optical window 21 and an imaging systemfor obtaining images from inside a body cavity or lumen, such as the GItract. The imaging system includes an illumination unit 23. Theillumination unit 23 may include one or more discrete light sources 23A,or may include only one light source 23A. The one or more light sources23A may be a white light emitting diode (LED), or any other suitablelight source, known in the art. The device 10A includes a CMOS imagingsensor 24, which acquires the images and an optical system 22 whichfocuses the images onto the CMOS imaging sensor 24. The illuminationunit 23 illuminates the inner portions of the body lumen through anoptical window 21. Device 10A further includes a transmitter 26 and anantenna 27 for transmitting the video signal of the CMOS imaging sensor24, and one or more power sources 25. The power source(s) 25 may be anysuitable power sources such as but not limited to silver oxidebatteries, lithium batteries, or other electrochemical cells having ahigh energy density, or the like. The power source(s) 25 may providepower to the electrical elements of the device 10A.

Typically, in the gastrointestinal application, as the device 10A istransported through the gastrointestinal (GI) tract, the imager, such asbut not limited to the multi-pixel CMOS sensor 24 of the device 10Aacquires images (frames) which are processed and transmitted to anexternal receiver/recorder (not shown) worn by the patient for recordingand storage. The recorded data may then be downloaded from thereceiver/recorder to a computer or workstation (not shown) for displayand analysis. Other systems and methods may also be suitable.

During the movement of the device 10A through the GI tract, the imagermay acquire frames at a fixed or at a variable frame acquisition rate.For example, the imager (such as, but not limited to the CMOS sensor 24of FIG. 1) may acquire images at a fixed rate of two frames per second(2 Hz). However, other different frame rates may also be used,depending, inter alia, on the type and characteristics of the specificimager or camera or sensor array implementation that is used, and on theavailable transmission bandwidth of the transmitter 26. The downloadedimages may be displayed by the workstation by replaying them at adesired frame rate. According to this implementation, the expert orphysician examining the data may be provided with a movie-like videoplayback, which may enable the physician to review the passage of thedevice through the GI tract.

One of the limitations of electronic imaging sensors is that they mayhave a limited dynamic range. The dynamic range of most existingelectronic imaging sensors is significantly lower than the dynamic rangeof the human eye. Thus, when the imaged field of view includes both darkand bright parts or imaged objects, the limited dynamic range of theimaging sensor may result in underexposure of the dark parts of thefield of view, or overexposure of the bright parts of the field of view,or both.

Various methods may be used for increasing the dynamic range of animager. Such methods may include changing the amount of light reachingthe imaging sensor, such as for example by changing the diameter of aniris or diaphragm included in the imaging device to increase or decreasethe amount of light reaching the imaging sensor, methods for changingthe exposure time, methods for changing the gain of the imager ormethods for changing the intensity of the illumination. For example, instill cameras, the intensity of the flash unit may be changed during theexposure of the film.

When a series of consecutive frames is imaged such as in video cameras,the intensity of illumination of the imaged field of view within thecurrently imaged frame may be modified based on the results ofmeasurement of light intensity performed in one or more previous frames.This method is based on the assumption that the illumination conditionsdo not change abruptly from one frame to the consecutive frame.

However, in an in vivo imaging device, for example, for imaging the GItract, which may operate at low frame rates and which is moved through abody lumen (e.g., propelled by the peristaltic movements of theintestinal walls), the illumination conditions may vary significantlyfrom one frame to the next frame. Therefore, methods of controlling theillumination based on analysis of data or measurement results ofprevious frames may not be always feasible, particularly at low framerates.

Therefore there is a need for an imaging device that provides moreaccurate illumination, possibly tailored to particular in-vivoillumination requirements or environmental conditions.

SUMMARY OF THE INVENTION

Some embodiments of the present invention include a device and methodfor operating an in vivo imaging device wherein the illuminationproduced by the device may be varied in intensity and/or durationaccording to, for example, the amount of illumination produced by thedevice, which is reflected back to the device. In such a manner, theillumination can be controlled and made more efficient.

According to some embodiments of the present invention, a method forimplementing light control in an in vivo device is provided.Accordingly, the parameters such as exposure time and/or the gainfactor, or other parameters, for transmitting the recorded light may bealtered. For example, the gain factor may be altered as a function of alight saturation level measured at at least one interval within theframe exposure period. In such a manner the in vivo device can preventcases of over and under exposure, in addition to helping to ensure thatexposure ceases after full exposure is attained.

According to some embodiments of the present invention, a method isprovided for detecting problematic pixels in an imaging device. Thismethod may enable defining and/or excluding problematic ornon-functional pixels, for example based on an initial short exposurethat enables a threshold saturation level to be reached only forproblematic pixels.

According to some embodiments of the present invention, a method isprovided for determining when an in vivo imaging device has entered aparticular part of a body. Accordingly, environmental measurementdevices may be used to detect environmental parameters, such as pHlevels and temperature levels etc. Results recorded from thesemeasurement devices may be used to define areas, regions, organs etc.wherein the in vivo device may be or may have been located. The devicemode may be changed in accordance with the resulting definition.

According to some embodiments of the present invention, a method isprovided for determining when an in vivo imaging device has entered abody, using dark frames. For example, when dark frames requiresubstantial gain factor to attain full exposure, the device may bedefined as being inside a body (a dark environment). The device mode maybe changed in accordance with the resulting definition.

BRIEF DESCRIPTION OF THE DRAWINGS

The present invention is herein described, by way of example only, withreference to the accompanying drawings, in which like components aredesignated by like reference numerals, it being understood that thesedrawings are given for illustrative purposes only and are not meant tobe limiting, wherein:

FIG. 1 is a schematic diagram illustrating an embodiment of a prior artautonomous in-vivo imaging device;

FIG. 2 is a schematic block diagram illustrating part of an in-vivoimaging device having an automatic illumination control system, inaccordance with an embodiment of the present invention;

FIG. 3 is a schematic cross-sectional view of part of an in-vivo imagingdevice having an automatic illumination control system and four lightsources, in accordance with an embodiment of the present invention;

FIG. 4 is a schematic front view of the device illustrated in FIG. 3;

FIG. 5 is a schematic diagram illustrating a method of timing of theillumination and image acquisition in an in vivo imaging device having afixed illumination duration, according to an embodiment of theinvention;

FIG. 6 is a schematic diagram illustrating one possible configurationfor an illumination control unit coupled to a light sensing photodiodeand to a light emitting diode, in accordance with an embodiment of thepresent invention;

FIG. 7 is a schematic diagram illustrating the illumination control unitof FIG. 6 in detail, in accordance with an embodiment of the presentinvention;

FIG. 8 is a schematic diagram useful for understanding a method oftiming of the illumination and image acquisition in an in vivo imagingdevice having a variable controlled illumination duration, according toan embodiment of the invention;

FIG. 9 is a schematic diagram useful for understanding a method oftiming of the illumination and image acquisition in an in vivo imagingdevice having a variable frame rate and a variable controlledillumination duration according to an embodiment of the invention;

FIG. 10A is a timing diagram schematically illustrating an imaging cycleof an in vivo imaging device using an automatic illumination controlmethod, in accordance with another embodiment of the present invention;

FIG. 10B is a schematic exemplary graph representing the light intensityas a function of time, possible when using a method of automaticillumination control, according to an embodiment of the invention, forexample as illustrated in FIG. 10A;

FIG. 10C is another exemplary schematic graph representing anotherexample of the light intensity as a function of time, possible whenusing a method of automatic illumination control, according to anembodiment of the invention, illustrated in FIG. 10A;

FIG. 11 is a schematic diagram illustrating an illumination control unitincluding a plurality of light sensing units for controlling a pluralityof light sources, in accordance with an embodiment of the presentinvention;

FIG. 12 is a schematic diagram illustrating a front view of anautonomous imaging device having four light sensing units and four lightsources, in accordance with an embodiment of the present invention;

FIG. 13 is a schematic top view illustrating the arrangement of pixelson the surface of a CMOS imager usable for illumination control, inaccordance with an embodiment of the present invention;

FIG. 14 is a schematic top view of the pixels of a CMOS imagerillustrating an exemplary distribution of control pixel groups suitablefor being used in local illumination control in an imaging device,according to an embodiment of the invention;

FIG. 15 is a schematic exemplary graph representing the light saturationas a function of pixel output and time, possibly when implementing lightcontrol, according to an embodiment of the invention;

FIG. 16A depicts a series of steps of a method according to anembodiment of the present invention;

FIG. 16B depicts a series of steps of a method according to an alternateembodiment of the present invention; and

FIG. 16C depicts a series of steps of a method according to anadditional embodiment of the present invention.

DETAILED DESCRIPTION OF THE INVENTION

Various aspects of the present invention are described herein. Forpurposes of explanation, specific configurations and details are setforth in order to provide a thorough understanding of the presentinvention. However, it will also be apparent to one skilled in the artthat the present invention may be practiced without the specific detailspresented herein. Furthermore, well known features may be omitted orsimplified in order not to obscure the present invention.

Some embodiments of the present invention are based, inter alia, oncontrolling the illumination provided by the in-vivo imaging devicebased on light measurement which is performed within the duration of asingle frame acquisition time or a part thereof.

It is noted that while the embodiments of the invention shownhereinbelow are adapted for imaging of the gastrointestinal (GI) tract,the devices and methods disclosed herein may be adapted for imagingother body cavities or spaces.

Reference is now made to FIG. 2 which is a schematic block diagramillustrating part of an in-vivo imaging device having an automaticillumination control system, in accordance with an embodiment of thepresent invention. The device 30 may be constructed as a swallowablevideo capsule as disclosed for the device 10A of FIG. 1 or in U.S. Pat.No. 5,604,531 to Iddan et al., or in Co-pending PCT Patent Application,Publication no. WO 01/65995 to Glukhovsky et al, both herebyincorporated by reference in their entirety. However, the system andmethod of the present invention may be used in conjunction with otherin-vivo imaging devices.

The device 30 may include an imaging unit 32 adapted for imaging the GItract. The imaging unit 32 may include an imaging sensor (not shown indetail), such as but not limited to the CMOS imaging sensor 24 ofFIG. 1. However, the imaging unit 32 may include any other suitable typeof imaging sensor known in the art. The imaging unit 32 may also includean optical unit 32A including one or more optical elements (not shown),such as one or more lenses (not shown), one or more composite lensassemblies (not shown), one or more suitable optical filters (notshown), or any other suitable optical elements (not shown) adapted forfocusing an image of the GI tract on the imaging sensor as is known inthe art and disclosed hereinabove with respect to the optical unit 22 ofFIG. 1.

The optical unit 32A may include one or more optical elements (notshown) which are integrated with the imaging unit 32, such as forexample, a lens (not shown) which is attached to, or mounted on, orfabricated on or adjacent to the imager light sensitive pixels (notshown) as is known in the art.

The device 30 may also include a telemetry unit 34 suitably connected tothe imaging unit 32 for telemetrically transmitting the images acquiredby the imaging unit 32 to an external receiving device (not shown), suchas but not limited to the receiver/recorder device disclosed in U.S.Pat. No. 5,604,531 to Iddan et al., or in Co-pending PCT PatentApplication, Publication No. WO 01/65995 to Glukhovsky et al.

The device 30 may also include a controller/processor unit 36 suitablyconnected to the imaging unit 32 for controlling the operation of theimaging unit 32. The controller/processor unit 36 may comprise anysuitable type of controller, such as but not limited to, an analogcontroller, a digital controller such as, for example, a data processor,a microprocessor, a micro-controller, or a digital signal processor(DSP). The controller/processor unit 36 may also comprise hybridanalog/digital circuits as are known in the art. Thecontroller/processor unit 36 may be suitably connected to the telemetryunit 34 for controlling the transmission of image frames by thetelemetry unit 34.

The controller/processor unit 36 may be (optionally) suitably connectedto the imaging unit 32 for sending control signals thereto. Thecontroller/processor unit 36 may thus (optionally) control thetransmission of image data from the imaging unit 32 to the telemetryunit 34.

The device 30 may include an illuminating unit 38 for illuminating theGI tract. The illuminating unit 38 may include one or more discretelight sources 38A, 38B, to 38N or may include only one light source;such light source(s) may be, for example, but are not limited to, thelight sources 23A of FIG. 1. The light source(s) 38A, 38B, to 38N of theilluminating unit 38 may be white light emitting diodes, such as thelight sources disclosed in co-pending PCT Patent Application,Publication No. WO 01/65995 to Glukhovsky et al. However, the lightsource(s) 38A, 38B, 38N of the illuminating unit 38 may also be anyother suitable light source, known in the art, such as but not limitedto incandescent lamp(s), flash lamp(s) or gas discharge lamp(s), or anyother suitable light source(s).

It is noted that, in accordance with another embodiment of the presentinvention, the in vivo imaging device may include a single light source(not shown).

The device 30 may also include an illumination control unit 40 suitablyconnected to the light sources 38A, 38B, to 38N of the illuminating unit38 for controlling the energizing of the light sources 38A, 38B, to 38Nof the illuminating unit 38. The illumination control unit 40 may beused for switching one or more of the light sources 38A, 38B, to 38N onor off, and/or for controlling the intensity of the light produced byone or more of the light sources 38A, 38B, to 38N, as is disclosed indetail hereinafter.

The controller/processor unit 36 may be suitably connected to theillumination control unit 40 for (optionally) sending control signalsthereto. Such control signals may be used for synchronizing or timingthe energizing of the light sources 38A, 38B, 38N within theilluminating unit 38, relative to the imaging cycle or period of theimaging unit 32. The illumination control unit 40 may be (optionally)integrated within the controller/processor unit 36, or may be a separatecontroller. In some embodiments, illumination control unit 40 and/orcontroller/processor unit 36 may be part of telemetry unit 34.

The device 30 may further include a light sensing unit(s) 42 for sensingthe light produced by the illuminating unit 38 and reflected from thewalls of the GI tract. The light sensing unit(s) 42 may comprise asingle light sensitive device or light sensor, or a plurality ofdiscrete light sensitive device(s) or light sensor(s), such as but notlimited to, a photodiode, a phototransistor, or the like. Other types oflight sensors known in the art and having suitable characteristics mayalso be used for implementing the light sensing unit or units ofembodiments of the present invention.

The light sensing unit(s) 42 may be suitably connected to theillumination control unit 40 for providing the illumination control unit40 with a signal representative of the intensity of the light reflectedfrom the walls of the gastrointestinal tract (or any other object withinthe field of view of the imaging unit 32). In operation, theillumination control unit 40 may process the signal received from thelight sensing unit(s) 42 and, based on the processed signal, may controlthe operation of the light source(s) 38A, 38B, to 38N as is disclosed indetail hereinabove and hereinafter.

The device 30 may also include a power source 44 for providing power tothe various components of the device 30. It is noted that for the sakeof clarity of illustration, the connections between the power source 44and the circuits or components of the device 30 which receive powertherefrom, are not shown in detail. The power source 44 may be, forexample, an internal power source similar to the power source(s) 25 ofthe device 10A, e.g., a battery or other power source. However, if thedevice 30 is configured as an insertable device (such as, for example,an endoscope-like device or a catheter-like device, or any other type ofin vivo imaging device known in the art), the power source 44 may alsobe an external power source which may be placed outside the device 30(such an external configuration is not shown in FIG. 2 for the sake ofclarity of illustration). In such an embodiment having an external powersource (not shown), the external power source (not shown) may beconnected to the various power requiring components of the imagingdevice through suitable electrical conductors (not shown), such asinsulated wires or the like.

It is noted that while for autonomous or swallowable in-vivo imagingdevice such as the device 10A the power source(s) 25 are preferably (butnot necessarily) compact power sources for providing direct current(DC), external power sources may be any suitable power sources known inthe art, including but not limited to power sources providingalternating current (AC) or direct current or may be power suppliescouples to the mains as is known in the art.

The various functions and processes implemented by the swallowablein-vivo imaging device may be executed by, for example, a processor unit(e.g., unit 36 in FIG. 2). These functions and processes may beimplemented by the processor unit 36 alone, and/or by alternative units,such as illumination control unit 40, telemetry unit 34, light sensingunits 42, imaging unit 32 etc., or any combination of units. The variousunits may optionally be integrated within the processor unit 36, suchthat the processor unit can be said to implement any of the functionsand processes herein described. The methods and processes described maybe also embodied in other sensing devices having other structures andother components.

Reference is now made to FIGS. 3 and 4. FIG. 3 is a schematiccross-sectional view of part of an in-vivo imaging device having anautomatic illumination control system and four light sources, inaccordance with an embodiment of the present invention. FIG. 4 is aschematic front view of the device illustrated in FIG. 3.

The device 60 (only part of which is shown in FIG. 3) includes animaging unit 64. The imaging unit 64 may be similar to the imaging unit32 of FIG. 2 or to the imaging unit 24 of FIG. 1. Preferably, theimaging unit 64 may be a CMOS imaging unit, but other different types ofimaging units may be also used. The imaging unit 64 may include CMOSimager circuitry, as is known in the art, but may also include othertypes of support and or control circuitry therein, as is known in theart and disclosed, for example, in U.S. Pat. No. 5,604,531 to Iddan etal., or in Co-pending PCT Patent Application, Publication No. WO01/65995 to Glukhovsky et al. The device 60 also includes an opticalunit 62 which may comprise a lens or a plurality of optical elements asdisclosed hereinabove for optical unit 22 of FIG. 1 and the optical unit32A of FIG. 2.

The device 60 may include an illuminating unit 63, which may includefour light sources 63A, 63B, 63C and 63D which may be disposed withinthe device 60 as shown in FIG. 4. The light sources 63A, 63B, 63C and63D may be the white LED light sources or as disclosed, for example, inCo-pending US patent application, PCT Patent Application, PublicationNo. WO 01/65995 to Glukhovsky et al., but may also be any other suitabletype of light sources, including but not limited to, infrared lightsources, monochromatic light sources, band limited light sources knownin the art or disclosed hereinabove.

It is noted that while in accordance with one embodiment of the presentinvention the light sources 63A, 63B, 63C and 63D are shown to beidentical, other embodiments of the invention may be implemented withmultiple light sources which may not be identical. Some of the lightsources may have a spectral distribution, which is different than thespectral distribution of the other light sources. For example, of thelight sources within the same device, one of the light sources may be ared LED, another light source may be a blue LED and another light sourcemay be a yellow LED. Other configurations of light sources are alsopossible.

The device 60 may also include a baffle 70, which may be conicallyshaped or which may have any other suitable shape. The baffle 70 mayhave an aperture 70A therein. The baffle 70 may be interposed betweenthe light sources 63A, 63B, 63C and 63D and the optical unit 62 and mayreduce the amount of light coming directly from the light sources 63A,63B, 63C and 63D to enter the aperture 70A. The device 60 may include atransparent optical dome 61 similar to the optical dome 21 of FIG. 1.The optical dome 61 may be made from a suitable transparent plasticmaterial or glass or from any other suitable material which issufficiently transparent to at least some of the wavelengths of lightproduced by the light sources 63A, 63B, 63C and 63D to allow foradequate imaging.

The device 60 may further include at least one light sensing unit 67 forsensing light, which is reflected from or diffused by the intestinalwall 76. The light sensing unit may be attached to the baffle 70 suchthat its light sensitive part 67A faces the optical dome 61. Preferably,but not necessarily, the light sensing unit 67 may be positioned on thesurface of baffle 70 at a position which allows the light sensing unit67 to sense an amount of light which is representative or proportionalto the amount of light entering the aperture 70A of the baffle 70. Thismay be true when the illuminated object is semi-diffusive (as theintestinal surface may be), and when the size of the light sensing unit67 and its distance from the imaging sensor axis 75 are small comparedto the diameter D of the capsule like device 60.

The device 60 (FIG. 3) is illustrated as being adjacent to theintestinal wall 76. In operation, light rays 72 which are generated bythe light sources 63A, 63B, 63C and 63D may penetrate the optical dome61 and may be reflected from the intestinal wall 76. Some of thereflected light rays 74 may reach the light-sensing unit 67. Otherreflected light rays (not shown) may reach the aperture 70A and pass theoptical unit 62 to be focused on the imaging unit 64.

The amount of light measured by the light-sensing unit 67 may beproportional to the amount of light entering the aperture 70A. Thus, themeasurement of the light intensity reaching the light sensing unit 67may be used to determine the light output of the light sources 63A, 63B,63C and 63D as is disclosed in detail hereinafter.

The device 60 also includes an illumination control unit 40A. Theillumination control unit 40A is suitably coupled to the light sensingunit 67 and to the illuminating unit 63. The illumination control unit40A may process the signal received from the light sensing unit 67 tocontrol the light sources 63A, 63B, 63C and 63D as is disclosed indetail hereinafter.

The device 60 may also include a wireless transmitter unit (not shown inFIG. 3) and an antenna (not shown in FIG. 3), such as but not limited tothe transmitter 26 and the antenna 27 of FIG. 1 or may include anysuitable telemetry unit (such as, but not limited to the telemetry unit34 of FIG. 2). The telemetry unit may be a transmitter or a transceiver,for wirelessly transmitting (and optionally also receiving) data andcontrol signals to (and optionally from) an external receiver/recorder(not shown in FIG. 3) as disclosed in detail hereinabove. The device 60may also include one or more power sources such as, for example, thepower sources 25 of FIG. 1, or any other suitable power sources, knownin the art.

Reference is now made to FIG. 5 which is a schematic diagramillustrating a method of timing of the illumination and imageacquisition in an in vivo imaging device having a fixed illuminationduration. The timing method may be characteristic for imaging deviceshaving CMOS imagers but may also be used in devices having other typesof imagers.

An image acquisition cycle or period starts at the time T. The firstimage acquisition cycle ends at time T1 and has a duration ΔT1. Thesecond image acquisition cycle starts at time T1, ends at time T2 andhas a duration ΔT1. Each imaging cycle or period may comprise two parts,an illumination period 90 having a duration ΔT2, and a dark period 92having a duration ΔT3. The illumination periods 90 are represented bythe hashed bars of FIG. 5. During the illumination period 90 of eachimaging cycle, the illumination unit (such as but not limited to theilluminating unit 38 of FIG. 2, or the illuminating unit 63 of FIG. 3)is turned on and provides light for illuminating the intestinal wall.During the dark period 92 of each imaging cycle, the illuminating unit(such as but not limited to the illuminating unit 38 of FIG. 2, or theilluminating unit 63 of FIG. 3) is switched off and does not providelight.

The dark period 92, or a part thereof, may be used for, for example,acquiring an image from the imager by, for example, scanning the pixelsof the imager and for processing the imager output signals and fortransmitting the output signals or the processed output signals to anexternal receiver or receiver/recorder device, as disclosed hereinabove.

It is noted that while for the sake of simplicity, the diagram of FIG. 5illustrates a case in which the image acquisition cycle duration isfixed, and imaging is performed at a fixed frame rate, this is notmandatory. Thus, the frame rate and therefore the image acquisitioncycle duration may vary during imaging in accordance with a measuredparameter such as, for example the velocity of the imaging device withinthe gastrointestinal tract.

Generally, different types of light control methods may be used forensuring adequate image acquisition.

In a first method, the amount of light impinging on the light sensingunit 67 may be continuously measured and recorded during theillumination of the target tissue by the illuminating unit 63 to providea cumulative value representative of the total cumulative number ofphotons detected by the light sensing unit 67. When this cumulativevalue reaches a certain value, the illuminating unit 63 may be shut offby switching off the light sources 63A, 63B, 63C, and 63D included inthe illuminating unit 63. In this way the device 60 may ensure that whenthe quantity of measured light is sufficient to result in an adequatelyexposed frame (on the average), the illuminating unit 63 is turned off.

One advantage of the first method is that if the light sources (such asthe light sources 63A, 63B, 63C, and 63D) are operated at their maximalor nearly maximal light output capacity, the switching off may saveenergy when compared to the energy expenditure in a fixed durationillumination period (such as the illumination period 90 of FIG. 5).

Another advantage of the first method is that it enables the shorteningof the duration of the illumination period within the imaging cycle incomparison with using a fixed illumination period. In a moving imagingdevice, such as the device 60, ideally, it may be desirable to have theillumination period as short as practically possible, since thisprevents or reduces image smearing due to the movement of the device 60within the GI tract. Thus, typically, in a moving imaging device, theshorter the illumination period, the sharper will the resulting image be(assuming that enough light is generated by the illuminating unit toensure adequate imager exposure).

This may be somewhat similar to the increasing of the shutter speed in aregular shutter operated camera in order to decrease the duration ofexposure to light to prevent smearing of the image of a moving object orimage, except that in embodiments of the present method there istypically no shutter and the illumination period is being shortenedcontrollably to reduce image smearing due to device movements in the GItract.

Reference is now made to FIGS. 6 and 7. FIG. 6 is a schematic diagramillustrating one possible configuration for an illumination control unitcoupled to a light sensing photodiode and to a light emitting diode, inaccordance with an embodiment of the present invention. FIG. 7 is aschematic diagram illustrating the illumination control unit of FIG. 6in detail, in accordance with an embodiment of the present invention.

The illumination control unit 40B of FIG. 6 may be suitably connected toa photodiode 67B, which may be operated as a light sensing unit. Anyother suitable sensing unit or light sensor may be used. Theillumination control unit 40B may be suitably connected to a lightemitting diode (LED) 63E. The LED 63E may be a white LED as disclosedhereinabove or may be any other type of LED suitable for illuminatingthe imaged target (such as the gastrointestinal wall). The illuminationcontrol unit 40B may receive a current signal from the photodiode 67B.The received signal may be proportional to the intensity of light(represented schematically by the arrows 81) impinging the photodiode67B. The illumination control 40B may process the received signal todetermine the amount of light that illuminated the photodiode 67B withinthe duration of a light measuring time period. The illumination control40B may control the energizing of the LED 63E based on the amount oflight that illuminated the photodiode 67B within the duration of thelight measuring time period. Examples of the type of processing andcontrol of energizing are disclosed in detail hereinafter. Theillumination control unit 40B may also receive control signals fromother circuitry components included in the in vivo imaging device. Forexample, the control signals may include timing and/or synchronizationsignals, on/off switching signals, reset signals, or the like.

The light sensing unit(s) and light producing unit(s) may be anysuitable light producing or sensing units other than diodes.

FIG. 7 illustrates one possible embodiment of the illumination controlunit 40B. The illumination control unit 40B may include, for example, anintegrator unit 80, a comparator unit 82 and a LED driver unit 84. Theintegrator unit 80 is coupled to the photodiode 67B to receive therefroma signal indicative of the intensity of the light impinging on thephotodiode 67B, and to record and integrate the amount of lightimpinging on the photodiode 67B. The integrator unit 80 may be suitablyconnected to the comparator unit 82.

The integrator unit 80 may record and integrate the amount of lightimpinging on the photodiode 67B, integrating the received signal, andoutput an integrated signal to the comparator unit 82. The integratedsignal may be proportional to or indicative of the cumulative number ofphotons hitting the photodiode 67B over the integration time period. Thecomparator unit 80 may be suitably connected to the LED driver unit 84.The comparator unit 80 may continuously compare the value of theintegrated signal to a preset threshold value. When the value of theintegrated signal is equal to the threshold value, the comparator unit82 may control the LED driver unit 84 to switch off the power to the LED63E and thus cease the operation of the LED 63E.

Thus, the illumination control unit 40A (of FIG. 3) may be constructedand operated similar to the illumination control unit 40B of FIGS. 6 and7.

It is noted that while the circuits illustrated in FIG. 7 may beimplemented as analog circuits, digital circuits and/or hybridanalog/digital circuits may be used in implementing the illuminationcontrol unit, as is disclosed in detail hereinafter (with respect toFIG. 11).

Reference is now made to FIG. 8, which is a schematic diagram useful forunderstanding a method of timing of the illumination and imageacquisition in an in vivo imaging device having a variable controlledillumination duration, according to one embodiment.

An image acquisition cycle or period starts at the time T. The firstimage acquisition cycle ends at time T1 and has a duration ΔT1. Thesecond image acquisition cycle starts at time T1, ends at time T2 andhas a duration ΔT1. In each imaging cycle, the time period having aduration ΔT4 defines the maximal allowable illumination period. Themaximal allowable illumination period ΔT4 may typically be a time periodwhich is short enough as to enable imaging without excessive imagesmearing or blurring due to the movement of the device 60 within the GItract. The time T_(M) is the time of the end of the maximal allowableillumination period ΔT4 relative to the beginning time of the firstimaging cycle.

The maximal allowable illumination period ΔT4 may be factory presettaking into account, inter alia, the typical or average (or maximal)velocity reached by the imaging device within the GI tract, (as may bedetermined empirically in a plurality of devices used in differentpatients), the type of the imaging sensor (such as, for example, theCMOS sensor 64 of the device 50) and its scanning time requirements, andother manufacturing and timing considerations. In accordance with oneimplementation of the invention, when imaging at 2 frames per secondΔT1=0.5 second, the duration of ΔT4 may be set to have a value in therange of 20-30 milliseconds. However, this duration is given by way ofexample only, and ΔT4 may have other different values. Typically, theuse of a maximal allowable illumination period ΔT4 of less than 30milliseconds may result in acceptable image quality of most of theacquired image frames without excessive degradation due to blurring ofthe image resulting from movement of the imaging device within the GItract.

The time period ΔT5 is defined as the difference between the entireimaging cycle duration ΔT1 and the maximal allowable illumination periodΔT4 (ΔT5=ΔT1−ΔT4).

At the time of beginning T of the first imaging cycle, the illuminationunit (such as but not limited to the illuminating unit 63 of FIG. 3) isturned on and provides light for illuminating the intestinal wall. Thelight sensing unit 67 senses the light reflected and/or diffused fromthe intestinal wall 76 and provides a signal to the illumination controlunit 40A of the device 60. The signal may be proportional to the averageamount of light entering the aperture 70A. The signal provided by thelight sensing unit 67 may be integrated by the illumination control unit40A as is disclosed in detail hereinabove with respect to theillumination control unit 40B of FIGS. 6 and 7.

The integrated signal may be compared to a preset threshold value (forexample by a comparator such as the comparator unit 82 of FIG. 7). Whenthe integrated signal is equal to the threshold value, the illuminationcontrol unit 40A ceases the operation of the light sources 63A, 63B, 63Cand 63D of the illuminating unit 63. The time TE1 is the time at whichthe illuminating control unit turns off the light sources 63A, 63B, 63Cand 63D within the first imaging cycle. The time interval beginning attime T and ending at time TE1 is the illumination period 94 (representedby the hashed bar labeled 94) for the first imaging cycle. Theillumination period 94 has a duration of ΔT6. It may be seen that forthe first imaging cycle ΔT6<ΔT4.

After the time TE1 the scanning of the pixels of the CMOS sensor 64 maybegin and the pixel data (and possibly other data) may be transmitted bythe transmitter (not shown in FIG. 3) or telemetry unit of the device60.

Preferably, the scanning (read out) of the pixels of the CMOS sensor 64may begin as early as the time TE1 of the termination of theillumination. For example the illumination control unit 40A may send acontrol signal to the CMOS sensor at time TE1 to initiate the scanningof the pixels of the CMOS sensor 64. However, the scanning of the pixelsmay also begin at a preset time after the time T_(M) which is the endingtime of the maximal allowable illumination period ΔT4, provided thatsufficient time is available for pixel scanning and data transmissionoperations. According to one embodiment, keeping the start of readouttime fixed, for example at T_(M), may enable simpler implementation ofthe receiving unit.

At the time of beginning T1 of the second imaging cycle, theilluminating unit 63 is turned on again. The light sensing unit 67senses the light reflected and/or diffused from the intestinal wall 76and provides a signal to the illumination control unit 40A of the device60. The signal may be proportional to the average amount of lightentering the aperture 70A.

The signal provided by the light sensing unit 67 may be integrated andcompared to the threshold value as disclosed hereinabove for the firstimaging cycle. When the integrated signal is equal to the thresholdvalue, the illumination control unit 40A turns off the light sources63A, 63B, 63C and 63D of the illuminating unit 63. However, in theparticular schematic example illustrated in FIG. 8, the intensity oflight reaching the light sensing unit 67 in the second imaging cycle islower than the intensity of light reaching the light sensing unit 67 inthe first imaging cycle.

This difference of the illumination intensity or intensity versus timeprofile between different imaging cycle may be due to, inter alia,movement of the device 60 away from the intestinal wall 76, or a changeof the position or orientation of the device 60 with respect to theintestinal wall 76, or a change in the light absorption or lightreflecting or light diffusion properties of the part of the intestinalwall 76 which is within the field of view of the device 60.

Therefore it takes longer for the integrated signal output of theintegrator unit to reach the threshold value. Therefore, theillumination control unit 40A turns the illuminating unit 63 off at atime TE2 (it is noted that TE2>TE1).

The time interval beginning at time T1 and ending at time TE2 is theillumination period 96 for the second imaging cycle. The illuminationperiod 96 (represented by the hashed bar labeled 96) has a duration ΔT7.It may be seen that for the second imaging cycle ΔT7<ΔT4.

Thus, the duration of the illumination period within different imagingcycles may vary and may depend, inter alia, on the intensity of lightreaching the light sensing unit 67.

After the time TE2 the scanning of the pixels of the CMOS sensor 64 maybegin and the pixel data (and possibly other data) may be transmitted asdisclosed in detail hereinabove for the first imaging cycle of FIG. 8.

It is noted that while for the sake of simplicity, the diagram of FIG. 8illustrates a case in which the image acquisition cycle duration ΔT1 isfixed, and imaging is performed at a fixed frame rate, this is notmandatory. Thus, the frame rate and therefore the image acquisitioncycle duration ΔT1 may vary during imaging in accordance with a measuredparameter such as, for example the velocity of the imaging device withinthe gastrointestinal tract. In such cases, the duration of the imagingcycle may be shortened or increased in response to the measured velocityof the device 60 in order to increase or decrease the frame rate,respectively.

For example, co-pending U.S. patent application Ser. No. 09/571,326,filed May 15, 2000, co-assigned to the assignee of the presentapplication, incorporated herein by reference in its entirety for allpurposes, discloses, inter alia, a device and method for controlling theframe rate of an in-vivo imaging device.

The automatic illumination control methods disclosed hereinabove may beadapted for use in device having variable frame rate. Such adaptationmay take into account the varying duration of the imaging cycle and theimplementation may depend, inter alia, on the amount of time required tocomplete the pixel scanning and the data transmission, the availableamount of power available to the device 60, and other considerations.

A simple way of adapting the method may be to limit the maximal framerate of the imaging device, such that even when the maximal frame rateis being used, there will be enough time left for pixel scanning anddata transmission within the time period.

Reference is now made to FIG. 9, which is a schematic diagram useful forunderstanding a method of timing of the illumination and imageacquisition in an in vivo imaging device having a variable frame rateand a variable controlled illumination duration.

The first imaging cycle of FIG. 9 is similar to the first imaging cycleof FIG. 8 except that the duration of the illumination period 98 of FIG.9 (represented by the hashed bar labeled 98) is longer than the durationof the illumination period 94 of FIG. 8. The first imaging cycle of FIG.9 starts at time T, ends at time T1, and has a duration ΔT1. The timeT_(M) represents the end of the maximal allowable illumination periodΔT4. The second imaging cycle of FIG. 9 begins at time T1 and ends attime T3. The duration of the second imaging cycle ΔT8 is shorter thanthe duration of the first imaging cycle ΔT1 (ΔT8<ΔT1). The duration ofthe second imaging cycle ΔT8 corresponds to the highest frame rateusable in the imaging device. The illumination period 100 of the secondimaging cycle (represented by the hashed bar labeled 100 of FIG. 9) istimed by the illumination control unit depending on the light intensityas disclosed in detail hereinabove. The time period 102 (represented bythe dotted bar labeled 102) represents the amount of time ΔT9 requiredfor scanning the pixels of the imager and transmitting the scanned framedata. T_(M) represents the time of ending of the maximal allowableillumination period relative to the beginning time of each imagingcycle. Thus, if the frame rate is increased, even at the highestpossible frame rate there is enough time to scan the pixels and transmitthe data.

It is noted that typically, in an exemplary in vivo imaging devicehaving a fixed frame rate, the time requited for scanning the pixels ofa CMOS sensor having approximately 66,000 pixels (such as but notlimited to a CMOS sensor arranged in a 256×256 pixel array), and fortransmitting the digital (serial) data signals to an external receiverrecorder may be approximately 0.4 seconds (assuming a scanning and datatransmission time of approximately 6 microseconds per pixel). Thus,assuming a maximal illumination period of approximately 20-30milliseconds, the frame rate may not be extended much higher than 2frames per second. Alternate frame rates may be used for example, forimplementing different readout rates.

It may however be possible to substantially shorten the time requiredfor scanning the pixels and for transmitting the data. For example, byincreasing the clock rate of the CMOS pixel array, it may be possible toreduce the time required to scan an individual frame to 3 microsecondsor even less. Additionally, it may be possible to increase the datatransmission rate of the transmitter 26 to even further shorten theoverall time required for scanning the array pixels for transmitting thepixel data to the external receiver/recorder.

Therefore, variable frame rate in vivo imaging devices, as well as fixedframe rate devices, may be implemented which may be capable of framerates of approximately 4-8 frames per second, and even higher.

When the method disclosed hereinabove for turning off the illuminatingunit when the integrated output of the light sensing unit reaches athreshold value adapted to ensure a good average image quality isimplemented, the tendency of the designer may be to operate theilluminating unit (such as, for example the illuminating unit 63 of FIG.3) close to the maximal available light output capacity. This may beadvantageous because of the shortened illumination period durationachievable which may improve image clarity by reducing movement inducedimage blurring.

It may not always be possible or desired to operate the illuminatingunit close to the maximal possible light output capacity. Therefore, itmay be desired to start the operation of the illuminating unit 63 at agiven light output which is lower than the maximal light output ofilluminating unit 63.

In a second illumination control method, the illuminating unit 63 ofFIG. 3 may be initially operated at a first light output level at thebeginning of each of the imaging cycles. The light sensing unit 67 maybe used to measure the amount of light during a short illuminationsampling period.

Reference is now made to FIGS. 10A, 10B and 10C. FIG. 10A is a timingdiagram schematically illustrating an imaging cycle of an in vivoimaging device using an automatic illumination control method inaccordance with another embodiment of the present invention. FIG. 10B isan exemplary schematic graph representing an example of the lightintensity as a function of time, possible when using the method ofautomatic illumination control illustrated in FIG. 10A. FIG. 10C is aschematic graph representing another example of the light intensity as afunction of time, possible when using the method of automaticillumination control illustrated in FIG. 10A.

In FIGS. 10A, 10B and 10C, the horizontal axes of the graphs representstime in arbitrary units. In FIGS. 10B and 10C, the vertical axisrepresents the intensity I of the light output by the illuminating unit63 (FIG. 3).

The automatic illumination control method illustrated in FIG. 10Aoperates by using an illumination sampling period 104 included in atotal illumination period 108. An imaging cycle 110 includes the totalillumination period 108 and a dark period 112. The illuminating unit 63may illuminate the intestinal wall 76 within the duration totalillumination period 108. The dark period 112 may be used for scanningthe pixels of the CMOS imager 64 and for processing and transmitting theimage data as disclosed in detail hereinabove.

The total illumination period of the imaging cycle starts at time T andends at time T_(M). The time T_(M) is fixed with respect to thebeginning time T of the imaging cycle 110, and represents the maximalallowable illumination time. Practically, the time T_(M) may be selectedto reduce the possibility of image blurring as explained hereinabove.For example, the time T_(M) may be selected as 20 milliseconds from thetime of beginning T of the imaging cycle 110 (in other words, theduration of the total illumination period 108 may be set at 30milliseconds), but other larger or smaller values of the time T_(M) andof the total illumination period 108 may also be used.

The total illumination period 108 may include an illumination samplingperiod 104 and a main illumination period 106. The illumination samplingperiod 104 starts at time T and ends at time T_(S). The mainillumination period 106 starts at time T_(S) and ends at time T_(M).

In an exemplary embodiment of the method, the duration of theillumination sampling period 104 may be set at approximately 2-5milliseconds, but other larger or shorter duration values may be useddepending, inter alia, on the type and characteristics of the lightsensing unit 67, its sensitivity to light, its signal to noise ratio(S/N), the intensity I₁ at which the illuminating unit 63 is operatedduring the illumination sampling period 104, and other implementationand manufacturing considerations.

Turning to FIGS. 10B and 10C, during the illumination sampling period104, the illuminating unit 63 is operated such that the intensity oflight is I₁. The light sensing 67 may sense the light reflected from anddiffused by the intestinal wall 76. The illumination control unit 40Amay integrate the intensity signal to determine the quantity Q of lightreaching the light sensing unit 67 within the duration of theillumination sampling period 104. The illumination control unit 40A maythen compute from the value Q and from the known duration of the mainillumination period 106, the intensity of light I_(N) at which theilluminating unit 63 needs to be operated for the duration of the mainillumination period 106 in order to provide adequate average exposure ofthe CMOS sensor 64. In one embodiment an estimated total amount of lightreceived is kept substantially constant across a set of imaging cycles,or is kept within a certain target range. The computation may beperformed, for example, by subtracting from a fixed light quantity whichis desired to be received or applied the amount of light recorded duringthe sampling period 104 and dividing the result by a fixed time periodwhich corresponds to the main illumination period 106. One possible wayto perform the computation would be using equation 1 as follows:I _(N)=(Q _(T) −Q)/ΔT _(MAIN)  equation 1

Wherein,

ΔT_(MAIN) is the duration of the main illumination period 106, Q_(T) isthe total quantity of light that needs to reach the light sensing unit67 within an imaging cycle to ensure adequate average exposure of theCMOS sensor 64, and Q is the quantity of light reaching the lightsensing unit 67 within the duration of an illumination sampling period104 of an imaging cycle.

It is noted that the value of Q_(T) may be empirically determined.

FIG. 10B schematically illustrates a graph showing the intensity oflight produced by the illuminating unit 63 as a function of time for anexemplary imaging cycle. During the illumination sampling period 104 thelight intensity has a value I₁. After the end of the illuminationsampling period 104, the light intensity I_(N)=I₂ may be computed asdisclosed in equation 1 hereinabove, or by using any other suitable typeof analog or digital computation.

For example, if the computation is digitally performed by thecontroller/processor 36 of FIG. 2, the value of I_(N) may be computedwithin a very short time (such as for example less than a microsecond)compared to the duration of the main illumination period 106.

If the computation of I_(N) is performed by an analog circuit (notshown) which may be included in the illumination control unit 40 of FIG.2, or in the illumination control unit 40B of FIG. 6, or in theillumination control unit 40A of FIG. 3, the computation time may alsobe short compared to the duration of the main illumination period 106.

After the computation of I₂ for the imaging cycle represented in FIG.10B is completed, the illumination control unit 40A may change theintensity of the light output of the illuminating unit of the imagingdevice to I₂. This may be achieved, for example, by increasing theamount of current output from the LED driver unit 84 of FIG. 7, or byincreasing the amount of current output from one or more LED driverunits (not shown in detail) which may be included in the illuminationcontrol unit 40A to supply current to the light sources 63A, 63B, 63C,and 63D. At the end of the main illumination period 108 (at time T_(M)),the illumination control unit 40A may switch the illuminating unit 63off until time T1 which is the beginning of a new imaging cycle (notshown). At the beginning of the new imaging cycle, the light intensityis switched again to the value I₁ and a new illumination sampling periodbegins.

FIG. 10C schematically illustrates a graph showing the intensity oflight produced by the illuminating unit 63 as a function of time foranother different exemplary imaging cycle. The illumination intensity I₁is used throughout the illumination sampling period 104 as disclosedhereinabove. In this imaging cycle, however, the value of Q measured forthe illumination sampling period 104 is higher than the value of Qmeasured for the illumination sampling period of FIG. 10B. This mayhappen, for example, due to movement of the position of the imagingdevice 60 relative to the intestinal wall 76. Therefore the computedvalue of I₃ is lower than the value of I₂ of the imaging cycleillustrated in FIG. 10B. The value of I₃ is also lower than the value ofI₁. Thus, the intensity of light emitted by the illuminating unit 63during the main illuminating period 106 illustrated in FIG. 10C is lowerthan the intensity of light emitted by the illuminating unit 63 duringthe illumination sampling period 104 of FIG. 10C.

It is noted that if the computed value of I₃ is equal to the value of I₁(case not shown in FIGS. 10B-10C) the illumination intensity may bemaintained at the initial value of I₁ for the duration of the totalillumination period 108, and no modification of the illuminationintensity is performed at time T_(M).

An advantage of the second illumination control method disclosedhereinabove may be that it may at least initially avoid the operating ofthe illuminating unit 63 at its maximal light output intensity. This maybe useful for improving the performance of the power sources, such as,for example, the power source(s) 25 of FIG. 1, and may extend the usefuloperational life thereof. It is known in the art that many batteries andelectrochemical cells do not perform optimally when they are operatednear their maximal current output. When using the second illuminationmethod, the light sources (such as the light sources 63A, 63B, 63C, and63D of FIG. 3) are initially operated at a light intensity I₁ which maybe a fraction of their maximal output light intensity. Thus, in caseswhere it is determined that the maximal light output intensity is notrequired for the current frame acquisition, the light sources may beoperated at a second light intensity level (such as, for example thelight intensity level I₃ which is lower than the light intensity levelI₁). Thus, the second illumination control method may reduce the currentrequired for operating the illuminating unit 63 drawn from the batteriesor other power sources of the imaging device which may extend the usefuloperational life of the batteries or of other power sources used in theimaging device. According to one embodiment a combination of bothmethods (variable duration and variable intensity) is possible.

It will be appreciated by those skilled in the art that the embodimentsof the present invention are not limited to the use of a single lightsensing element and/or a single light source. Additionally, it will beappreciated that the light sensing elements may include photo detectorsthat are separate from an imager, or are part of an imager.

Reference is now made to FIG. 11 which is a schematic diagramillustrating an illumination control unit for controlling a plurality oflight sources, in accordance with an embodiment of the presentinvention.

The illumination control unit 120 includes a plurality of light sensingunits 122A, 122B, . . . 122N, suitably interfaced with a plurality ofanalog to digital (A/D) converting units 124A, 124B, . . . 124N,respectively. The A/D converting units are suitably connected to aprocessing unit 126. The processing unit 126 is suitably connected to aplurality of LED drivers 128A, 128B, . . . 128N which are suitablyconnected to a plurality of LED light sources 130A, 130B, . . . 130N.

Signals representing the intensity of light sensed by the light sensingunits 122A, 122B, . . . 122N are fed to the A/D converting units 124A,124B, . . . 124N, respectively, which output digitized signals. Thedigitized signals may be received by the processing unit 126 which mayprocess the signals. For example the processing unit 136 may performintegration of the signals to compute the quantity of light sensed byone or more of the light sensing units 122A, 122B, . . . 122N. Thecomputed quantity of light may be the total combined quantity of lightsensed by all the light sensing units 122A, 122B, . . . 122N takentogether, or may be the individual quantities of light separatelycomputed for each individual light sensing unit of the light sensingunits 122A, 122B, . . . 122N.

The processing unit 136 may further process the computed light quantityor light quantities, to provide control signals to the LED drivers 128A,128B, . . . 128N which in turn may provide, individually or incombination, suitable currents to the LED light sources 130A, 130B, . .. 130N. According to one embodiment of the present invention, eachsensor may be directly related to one or more illumination sources.

According to some embodiments of the present invention, individualcontrol of illumination sources may be enabled by using special controlpixels. These control pixels may be adapted for fast read-out, which iswell known in the art. A fast read-out procedure may not reset the pixelvalues.

It is noted that the illumination control unit 120 of FIG. 11 may beoperated using different processing and control methods.

In accordance with one embodiment of the present invention, all thelight sensing units 122A, 122B, . . . 122N may be used as a single lightsensing element and the computation is performed using the combinedtotal quantity of light to simultaneously control the operation of allthe LED light sources 130A, 130B, . . . 130N together. In thisembodiment, the illumination control unit 120 may be implemented using,for example, the first illumination control method as disclosedhereinabove and illustrated in FIGS. 5, 8, and 9, which uses a fixedillumination intensity and computes the termination time of theillumination. According to other embodiments multiple A/D units (e.g.,124) are not included, rather analog processing is performed.

Alternatively, in accordance with another embodiment of the presentinvention, the illumination control unit 120 may be implemented usingthe second illumination control method, for example, as disclosedhereinabove and illustrated in FIGS. 10A-10C which uses a firstillumination intensity I₁ in an illumination sampling period andcomputes a second light intensity I_(N) for use in a main illuminationperiod as disclosed in detail hereinabove. In such a case, theillumination intensity I₁ used throughout the illumination samplingperiod 104 (see FIGS. 10A-10C) may be identical for all the LED lightsources 130A, 130B, . . . 130N, and the illumination intensity I_(N)used throughout the main illumination period 106 (FIGS. 10A-10C) may beidentical for all the LED light sources 130A, 130B, . . . 130N.

In accordance with another embodiment of the present invention, each ofthe light sensing units 122A, 122B, . . . 122N may be used as a separatelight sensing unit and the computation may be performed using theindividual quantities of light sensed by each of the light sensing units122A, 122B, . . . 122N to differentially control the operation of atleast one of the LED light sources 130A, 130B, . . . 130N respectivelyor in any combination. In this embodiment, the illumination control unit120 may be implemented using the first illumination control method asdisclosed hereinabove and illustrated in FIGS. 5, 8, and 9, which uses afixed illumination intensity for each of the LED light sources 130A,130B, . . . 130N and may separately compute the termination time of theillumination for each of the LED light sources 130A, 130B, . . . 130N.In such a manner, sets of light sources 130A, 130B, . . . 130N (where aset may include one) may be paired with sets of sensors 122A, 122B, . .. 122N.

Alternatively, in accordance with another embodiment of the presentinvention, the illumination control unit 120 may be implemented usingthe second illumination control method as disclosed hereinabove andillustrated in FIGS. 10A-10C which uses a first illumination intensityI₁ in an illumination sampling period and computes a second lightintensity I_(N) for use in a main illumination period as disclosed indetail hereinabove. In such a case, the illumination intensity I₁ may beidentical for all the LED light sources 130A, 130B, . . . 130N, and theillumination intensity I_(N) may be identical for all the LED lightsources 130A, 130B, . . . 130N.

Typically, this embodiment may be used in cases in which the positioningof the light sources 130A, 130B, . . . 130N and the light sensing units122A, 122B, . . . 122N in the imaging device is configured to ensurethat a reasonably efficient “local control” of illumination is enabledand that the cross-talk between different light sources is at asufficiently low level to allow reasonable local control of theillumination intensity produced by a one or more of the light sources130A, 130B, . . . 130N by processing the signals from one or more lightsensing unit which are associated in a control loop with the one or morelight sources.

Reference is now made to FIG. 12 which is a schematic diagramillustrating a front view of an autonomous imaging device having fourlight sensing units and four light sources, in accordance with anembodiment of the present invention.

The device 150 includes four light sources 163A, 163B, 163C and 163D andfour light sensing units 167A, 167B, 167C and 167D. The light sources163A, 163B, 163C and 163D may be the white LED sources as disclosedhereinabove, or may be other suitable light sources. The light sensingunits 167A, 167B, 167C and 167D are attached on the surface of thebaffle 70, surrounding the aperture 62. The front part of the device 150may include four quadrants 170A, 170B, 170C and 170D. The device 150 mayinclude an illumination control unit (not shown in the front view ofFIG. 12), and all the optical components, imaging components, electricalcircuitry, and power source(s) for image processing and transmitting asdisclosed in detail hereinabove and illustrated in the drawing Figures(See FIGS. 1, 2).

The quadrants are schematically represented by the areas 170A, 170B,170C and 170D between the dashed lines. In accordance with an embodimentof the invention, the device 150 may include four independent localcontrol loops. For example, the light source 163A and the light sensingunit 167A which are positioned within the quadrant 170A may be suitablycoupled to the illumination control unit (not shown) in a way similar tothe coupling of the light sources 38A-38N and the light sensing unit(s)42 to the illumination control unit 40 of FIG. 2. The signal from thelight sensing unit 167A may be used to control the illuminationparameters of the light source 163A using any of the illuminationcontrol methods disclosed hereinabove, forming a local control loop forthe quadrant 170A.

Similarly, the signal from the light sensing unit 167B may be used tocontrol the illumination parameters of the light source 163B using anyof the illumination control methods disclosed hereinabove, forming alocal control loop for the quadrant 170B, the signal from the lightsensing unit 167C may be used to control the illumination parameters ofthe light source 163C using any of the illumination control methodsdisclosed hereinabove, forming a local control loop for the quadrant170C, and the signal from the light sensing unit 167D may be used tocontrol the illumination parameters of the light source 163D using anyof the illumination control methods disclosed hereinabove, forming alocal control loop for the quadrant 170D.

It is noted that there may be some cross-talk or interdependency betweenthe different local control loops, since practically, some of the lightproduced by the light source 163A may be reflected from or diffused bythe intestinal wall and may reach the light sensing units 167B, 167C,and 167D which form part of the other local control loops for the otherquadrants 170B, 170C, and 170D, respectively.

The arrangement of the positions light sensing units 167A, 167B, 167Cand 167D and the light sources 163A, 163B, 163C and 163D within thedevice 150 may be designed to reduce such cross-talk.

In other embodiments of the invention it may be possible to useprocessing methods such as “fuzzy logic” methods or neural networkimplementations to link the operation of the different local controlloops together. In such implementations, the different local controlloops may be coupled together such that information from one of thelight sensing unit may influence the control of illumination intensityof light sources in other local control loops.

It is noted that, while the imaging device 150 illustrated in FIG. 12includes four light sources and four light sensing units, The number oflight sources may vary and the imaging device of embodiments of thepresent invention may be constructed with a different number (higher orlower than four) of light sources. Similarly, the number of the lightsensing units may also vary, and any suitable or practical number oflight sensing units may be used. Additionally, it is noted that thenumber of light sensing units in a device need not be identical to thenumber of light sources included in the device. Thus, for example, adevice may be constructed having three light sensing units and six lightsources. Or in another example, a device may be constructed having tenlight sensing units and nine light sources.

The factors determining the number of light sources and the number oflight sensing units may include, inter alia, the geometrical (twodimensional and three dimensional) arrangement of the light sources andthe light sensing units within the device and their arrangement relativeto each other, the size and available power of the light sources, thesize and sensitivity of the light sensing units, manufacturing andwiring considerations.

The number of local control loops may also be determined, inter alia, bythe degree of uniformity of illumination desired, the degree ofcross-talk between the different local control loops, the processingpower of the illumination control unit available, and othermanufacturing considerations.

The inventors of the present invention have noticed that it is alsopossible to achieve illumination control using one or more of the lightsensitive pixels of the imager itself, instead of or in addition tousing dedicated light sensing unit(s) which are not part of the imager.In addition, special light sensing elements may be integrated into thepixel array on the surface of the CMOS imager IC.

For example, in CMOS type imagers, some of the pixels of the CMOS imagermay be used for controlling the illumination, or alternatively,specially manufactured light sensitive elements (such as, analogphotodiodes, or the like) may be formed within the pixel array of theimager.

Reference is now made to FIG. 13 which is a top view schematicallyillustrating the arrangement of pixels on the surface of a CMOS imagerusable for illumination control, in accordance with an embodiment of thepresent invention. It is noted that the pixel arrangement in FIG. 13 isonly schematically illustrated and the actual physical arrangement ofthe circuitry on the imager is not shown.

The surface of the CMOS imager 160 is schematically represented by a12×12 array comprising 144 square pixels. The regular pixels 160P areschematically represented by the white squares. The CMOS imager alsoincludes sixteen control pixels 160C, which are schematicallyrepresented by the hashed squares.

It is noted that while the number of the pixels in the CMOS imager 160was arbitrarily chosen as 144 for the sake of simplicity and clarity ofillustration only, the number of pixels may be larger or smaller ifdesired. Typically, a larger numbers of pixels may be used to provideadequate image resolution. For example a 256×256 pixel array may besuitable for GI tract imaging.

In accordance with an embodiment of the present invention, the controlpixels 160C may be regular CMOS imager pixels which are assigned to beoperated as control pixels. In accordance with this embodiment, thecontrol pixels 160C may be scanned at a different time than the regularimaging pixels 160P. This embodiment has the advantage that it may beimplemented with a regular CMOS pixel array imager.

Turning back to FIG. 10A, the timing diagram of FIG. 10A may also beused, according to one embodiment, to illustrate the automaticillumination control method using control pixels. The method may operateby using a fast scan of the control pixels 160C at the beginning of eachimaging cycle 110. The illuminating unit (not shown) may be turned on atthe beginning of the imaging cycle 110 (at time T). The scanning of thecontrol pixels 160C may be performed similar to the scanning of theregular pixels 160P, except that the scanning of all of the controlpixels 160C occurs within the illumination sampling period 104. Thecontrol pixels 160C may be serially scanned within the duration of theillumination sampling period 104. This is possible due to the ability torandomly scan any desired pixel in a CMOS pixel array, by suitablyaddressing the pixel readout lines (not shown) as is known in the art.

It is noted that since the control pixels 160C are scanned serially (oneafter the other), the control pixel which is scanned first has beenexposed to light for a shorter time period than the control pixels whichare scanned next. Thus, each control pixel is scanned after it has beenexposed to light for a different exposure time period.

If one assumes that the intensity of light reflected from the intestinalwall does not change significantly within the duration of theillumination sampling period 104, it may be possible to compensate forthis incrementally increasing pixel exposure time by computationallycorrecting the average measured light intensity for all the controlpixels 160C, or the computed average quantity of light reaching all thecontrol pixels 160C. For example, a weighted average of the pixelintensities may be computed.

Alternatively, in accordance with another embodiment of the presentinvention, the illuminating unit 63 may be turned off after the end onthe illumination sampling period 104 (the turning off is not shown inFIG. 10A). This turning off may enable the scanning of the controlpixels 160C while the pixels 160C are not exposed to light and may thusprevent the above described incremental light exposure of the controlpixels.

After the scanning (readout) of all the control pixels 160C is completedand the scanned control pixel signal values are processed (by analog orby digital computation or processing), the value of the requiredillumination intensity in the main illumination period may be computed,for example, by the illumination control unit 40A (or, for example, bythe illumination control unit 40 of FIG. 2).

The computation of the required illumination intensity or of the currentrequired from the LED driver unit 84 may be performed as disclosedhereinabove, using the known value of I₁ (see FIG. 10B) and may or maynot take into account the duration of the period in which theilluminating unit 63 was turned off (this duration may be approximatelyknown from the known time required to scan the control pixels 160C andfrom the approximate time required for the data processing and/orcomputations). The illumination unit 63 may then be turned on (theturning on is not shown in FIG. 10A for the sake of clarity ofillustration) using the computed current value to generate the requiredillumination intensity value I₂ (see FIG. 10B) till the end of the mainillumination period 106 at time T_(M).

It is noted that if the number of control pixels 160C is small the timerequired for scanning the control pixels 160C may be short in comparisonto the total duration of the total illumination period 108. For example,if the scan time for scanning a single control pixel is approximately 6microseconds, the scanning of 16 control pixels may require about 96microseconds. Since the time required for computing the required lightintensity may also be small (a few microseconds or tens of microsecondsmay be required), the period of time during which the illumination unit63 is turned off at the end of the illumination sampling period 104 maycomprise a small fraction of the main illumination period 108 which maytypically be 20-30 milliseconds.

It may also be possible to compute a weighted average in which theintensity read for each pixel may be differently weighted according tothe position of the particular control pixel within the entire pixelarray 160. Such weighting methods may be used for obtaining centerbiased intensity weighting, as is known in the art, or any other type ofbiased measurement known in the art, including but not limited to edge(or periphery) biased weighting, or any other suitable type of weightingknown in the art. Such compensating or weighting computations may beperformed by an illumination control unit (not shown) included in theimaging device, or by any suitable processing unit (not shown), orcontroller unit (not shown) included in the imaging device in which theCMOS imager 160 illustrated in FIG. 13 is included.

Thus, if an averaging or weighting computation is used, after thereadout of the control pixels and any type of compensation or weightingcomputation is finished, the illumination control unit (not shown) maycompute the value of the weighted (and/or compensated) quantity of lightsensed by the control pixels 160C and use this value for computing thevalue of I₂.

It is noted that the ratio of the number of the control pixels 160C tothe regular pixels 160P should be a small number. The ratio of 16/144which is illustrated is given by example only (for the sake of clarityof illustration). In other implementations the ratio may be differentdepending, inter alia, on the total number of pixels in the CMOS arrayof the imager and on the number of control pixels used. For example in atypical 256×256 CMOS pixel array it may be practical to use 16-128pixels as illumination control pixels for illumination control purposes.The number of control pixels in the 256×256 CMOS pixel array may howeveralso be smaller than 16 control pixels or larger than 128 controlpixels.

Generally, the number of control pixels and the ratio of control pixelsto regular pixels may depend, inter alia, on the total number of pixelsavailable on the imager pixel array, on the pixel scanning speed of theparticular imager, on the number of control pixels which may bepractically scanned in the time allocated for scanning, and on theduration of the illumination sampling period.

An advantage of the embodiments using automatic illumination controlmethods in which some of the pixels of the CMOS imager pixel array (suchas for example the example illustrated in FIG. 13) is that in contrastto light sensitive sensors which may be disposed externally to thesurface of the imager (such as for example, the light sensing unit 67 ofFIG. 3), the control pixels 160C actually sense the amount of lightreaching the imager's surface since they are also imaging pixelsdisposed on the surface of the imager. This may be advantageous due to,inter alia, higher accuracy of light sensing, and may also eliminate theneed for accurately disposing or the light sensing unit at an optimalplace in the optical system, additionally, the control pixels may havesignal to noise characteristics and temperature dependence propertiessimilar to the other (non-control) pixels of the imager.

Another advantage of using control pixels is that no external lightsensing units are needed which may reduce the cost and simplify theassembly of the imaging device.

It is noted that, according to one embodiment, in a CMOS imager such asimager 160, the scanning of the control pixels 160C after theillumination sampling period 104 does not reset the pixels. Thus, thecontrol pixels 160C continue to sense and integrate the light during themain illumination period 106, and are scanned after the time T_(M)together with all the other regular pixels 160P of the imager 160. Thus,the acquired image includes the full pixel information since the controlpixels 160C and the regular pixels 160P have been exposed to light forthe same duration. The image quality or resolution is thus notsignificantly affected by the use of the control pixels 160C forcontrolling the illumination.

It is also noted that while the arrangement of the control pixels 160Con the imager 160 is symmetrical with respect to the center of theimager, any other suitable arrangement of the pixels may be used. Thenumber and the distribution of the control pixels on the imager 160 maybe changed or adapted in accordance with the type of averaging usedand/or for example, with the type of acquired images.

Furthermore, the control pixels may be grouped into groups that may beprocessed separately to allow local illumination control in imagersusing a plurality of separately controllable light sources.

Reference is now made to FIG. 14, which is a schematic top view of thepixels of a CMOS imager illustrating an exemplary distribution ofcontrol pixel groups suitable for being used in local illuminationcontrol in an imaging device, in accordance with an embodiment of thepresent invention.

The illustrated imager 170 is a 20×20 pixel array having 400 pixels. Thecontrol pixels are schematically represented by the hashed squares 170A,170B, 170C and 170D and the remaining imager pixels are schematicallyrepresented by the non-hashed squares 170P. Four groups of controlpixels are illustrated on the imager 170.

The first pixel group includes four control pixels 170A arranged withinthe top left quadrant of the surface of the imager 170. The second pixelgroup includes four control pixels 170B arranged within the top rightquadrant of the surface of the imager 170. The third pixel groupincludes four control pixels 170C arranged within the bottom rightquadrant of the surface of the imager 170. The fourth pixel groupincludes four control pixels 170D arranged within the top left bottomquadrant of the surface of the imager 170.

If the imager 170 is disposed in an autonomous imaging device having aplurality of light sources (such as, but not limited to the device 150of FIG. 12), each of the four groups of control pixels 170A, 170B, 170Cand 170D may be scanned and processed as disclosed hereinabove toprovide data for locally controlling the illumination level reachingeach of the respective four quadrants of the imager 170. The scanneddata for each of the pixels within each of the four groups may beprocessed to compute a desired value of illumination intensity for therespective imager quadrant. The methods for controlling the illuminationusing separate local control loops may be similar to any of the methodsdisclosed hereinabove with respect to the device 150 of FIG. 12, exceptthat in the device 150 the light sensing units are units external to theimager and in the device 170, the control pixels used for sensing areimager pixels which are integral parts of the imager 170.

The illumination control methods using control pixels may implementedusing the closed-loop method of terminating the illumination when theintegrated sensor signal reaches a threshold level as disclosedhereinabove or may be implemented by using an initial illuminationintensity in a sampling illumination period and adapting or modifyingthe illumination intensity (if necessary) in accordance with a valuecomputed or determined from the control pixel scanning as disclosedhereinabove.

The signals or data of (representing the pixel charge) the pixel groupsmay be processed using averaging or weighted averaging methods toperform center biased or periphery biased averages or according to anyother averaging or processing method known in the art. The results ofthe processing may be used as disclosed hereinabove to control the lightsources (such as for example four light sources disposed within theimaging device in an arrangement similar to the arrangement of the fourlight sources 163A, 163B, 163C, and 163D of FIG. 12).

It will be appreciated by those skilled in the art that the number ofcontrol pixels and the distribution of the control pixels on the surfaceof the imager may be varied, inter alia, in accordance with the desiredtype of averaging, the required number of local illumination controlgroups, the number and position of the light sources available in theimaging device, the computational power available to the processing unitavailable, the speed of the illumination control unit, and other designconsiderations.

In accordance with another embodiment of the present invention, thecontrol pixels 160C of FIG. 13 may be specially fabricated pixels, whichare constructed differently than the regular pixels 160P. In accordancewith this embodiment, the control pixels 160C may be fabricated asanalog photodiodes with appropriate readout or sampling circuitry (notshown) as is known in the art. This implementation may use a speciallyfabricated custom CMOS imager in which the analog photodiodes serving asthe control pixels 160C may be read simultaneously which may beadvantageous since the readout or scanning time may be shorter than thetime required to sequentially scan the same number of control pixelsimplemented in a regular CMOS pixel array having uniform pixelconstruction.

It is noted that when analog photodiodes or other known types ofdedicated sensors are integrated into the CMOS pixel array of theimaging device, the acquired image will have “missing” image pixels,since the area in which the analog photodiode is disposed is not scannedtogether with the regular CMOS array pixels. The image data willtherefore have “missing pixels”. If, however, a small number of analogphotodiodes or other dedicated control pixels is included in the CMOSpixel array, the missing pixels may not cause a significant degradationof image quality. Additionally, such dedicated analog photodiodes orother control pixels may be distributed within the pixel array and maybe sufficiently spaced apart from each other, so that image quality maybe only slightly affected by the missing image pixels.

It is noted that while the illumination control methods are disclosedfor use in an autonomous imaging device such as the device 10A of FIG.1, these illumination control methods may also be used with or withoutadaptations in other in-vivo imaging devices having an imager and anillumination unit, such as in endoscopes or catheter-like devices havingimaging sensor arrays, or in devices for performing in-vivo imagingwhich are insertable through a working channel of an endoscope, or thelike.

Additionally, the illumination control methods disclosed herein may beused in still cameras and in video cameras which include a suitableimager, such as a CMOS imager, and which include or are operativelyconnected to an illumination source.

Additionally, the use of control pixels implemented in a CMOS pixelarray imagers, using selected regular pixels as control pixels or usingspecially fabricated control pixels such as the analog photodiodes orthe like, may be applied for controlling the illumination of a flashunit or another illumination unit which may be integrated within thecamera or may be external to the camera and operatively connectedthereto.

The advantages of using control pixels which are part of the CMOS imagerof the camera may include, inter alia, simplicity of construction andoperation, the ability to implement and use a plurality of controllablyinterchangeable averaging methods including weighted averaging methodsand biasing methods, as disclosed in detail hereinabove, increasedaccuracy of illumination control.

Additionally, in specialty cameras operating under conditions in whichthe light source included in the camera or operatively connected theretois the only source of available illumination (such as, for example, incamera's operated at the bottom of the ocean, or in cameras which aredesigned to perform surveillance or monitoring in difficult to accessareas which are normally dark), the use of illumination control methodsdisclosed hereinabove may allow to use shutterless cameras, which mayadvantageously increase the reliability of such devices, reduce theircost, and simplify their construction and operation.

It is noted that, while in the embodiments of the invention disclosedhereinabove the number and the arrangement of the control pixels arefixed, in accordance with another different embodiment of the presentinvention, the number and/or the geometrical configuration (arrangement)of the control pixels may be dynamically changed or controlled. Forexample, briefly turning to FIG. 2, the light sensing unit(s) 42 mayrepresent one or more control pixels of a CMOS pixel array, and theillumination control unit 40, and/or the controller/processor unit 36may be configured for changing the number of the control pixels used inan imaging acquisition cycle and/or for changing the arrangement of thecontrol pixels on the pixel array of the imaging unit 32.

Such changing of control pixel number and/or arrangement may beperformed, in a non-limiting example, by changing number and/orarrangement of the pixels selected to be scanned as control pixelsduring the illumination sampling period 104 (FIG. 10A). Such a changingmay allow the use of different averaging arrangements and methods andmay allow changing of different biasing methods for different imagingcycles.

Additionally, using dynamically controllable control pixelconfiguration, it may be possible to implement two or more illuminationsampling periods within a single imaging cycle and to use a differentpixel number or configuration for each of these two or more illuminationsampling periods.

It may also be possible to remotely control the number and/orconfiguration of the control pixels, by instructions which arewirelessly transmitted to the telemetry unit, for example, telemetryunit 34 (FIG. 2), in which case the telemetry unit may be configured asa transceiver unit capable of transmitting data and of receiving controldata transmitted to it by an external transmitter unit (not shown inFIG. 2).

It is noted that, while the embodiments disclosed hereinabove were basedon modifying the light output from the illumination unit(s) (such as,for example the illumination unit 63 of FIG. 3) based on measurement andprocessing of the amount of light reaching the light sensing elements(such as, for example the light sensing unit 67 of FIG. 3, or the lightsensing units 42 of FIG. 2, or the control pixels 160C of FIG. 13),another approach may be used.

According to some embodiments of the present invention, the gain of thepixel amplifiers of the imager (for example, imaging unit 32 of FIG. 2)and/or other parameters may be changed. The parameter change decisionsmay be based on, for example, the results of the measurement of theamount of light reaching the light sensing unit or units (such as, forexample, the light sensing unit 42 of FIG. 2, the control pixels 160C ofFIG. 13, or the like). In such an embodiment, the illumination unit ofthe imaging device(s) (such as, for example, the illumination unit 63Aof FIG. 3, or the illumination unit 38 of FIG. 2 etc.) may be operatedat a fixed or variable illumination intensity, for a fixed or variabletime period. The light that reaches the light sensing unit(s) or thecontrol pixels of the imaging device during a sampling period (e.g., aportion of the exposure period within a frame) may be measured at asampling instance or point. A sampling instance or point may be adiscrete point in time or may extend over a certain amount of time.Parameters such as the gain level or sensitivity of the imager pixelamplifiers, the light intensity, duration of illumination or otherparameters may be changed individually or in any combination in relationto, for example, measurements of relative light saturation of selectedpixels, to achieve proper or appropriate imaging.

For example, if the amount of light reaching the light sensing unit(s)during an illumination sampling period, as measured at at least oneselected sampling instance (period), is approximately sufficient toensure proper image exposure (relative to, for example, an expectedthreshold for a determined number of pixels), the exposure may bestopped. In this case full exposure has already been achieved, and nofurther exposure is necessary. In this case, no change is the currentimage gain level should be required when transmitting the image. Inaddition, since the exposure was relatively short, there should be noproblem with blurring, which may accompany images recorded with longlight exposures.

In the case where the measurement at a sampling instance determines thattoo little saturation has been attained (relative to a threshold value)during an illumination sampling period, the exposure should be continuedto enable sufficient lighting for an image. However, too much exposuremay cause blurring, so the imager may be commanded to lower thesaturation threshold so as to have a shorter exposure. In addition tolowering the saturation threshold, in order to compensate for the shortexposure, the imager may be commanded to provide a higher gain level forthe image transmission, to enable enough exposure in spite of the shortexposure time. For example, if the saturation threshold was halved toshorten the exposure sufficiently, the gain level will correspondinglyneed to be doubled to enable adequate exposure.

If too much light, possibly relative to an expected threshold, reachesthe light sensing unit(s) during an illumination sampling period, theexposure may be stopped, and the pixel amplifier gain (or otherparameters) may be decreased to prevent overexposure. Additionalsampling periods may be instituted at chosen instances, to enablefurther fine-tuning of variables such as image gain and exposure time.

In addition to changes in the analog gain, which may be based oncontinuously scanning the analog output of a selected number of pixelsduring the early phase of the exposure period, the exposure may bestopped at any stage where full (e.g., adequate) saturation has beenreached. In this way, for example, full image exposure may be providedin many cases of low exposure, by, for example, adding gain levels toimages. In addition, over exposure can be prevented in many cases wherethere is high exposure, by, for example stopping exposure whensaturation is attained. These changes may result in increased imagequality, energy saving and/or other benefits.

Various embodiments may utilize various time, saturation, and voltagelevels, and are not limited to the following defined levels. Accordingto a particular application of the present embodiment, required timeresolution, which defines the maximum read out time required to achievesaturation for all pre-selected pixels may be, for example, 0.25 s.Other values or ranges may be used.

According to some embodiments of the present invention, a total exposuretime (e.g., an expected time required for adequate and/or correctexposure) may be defined (T1), within which an exposure measurement time(sampling time, such as T1/₄ or other times which are a portion of T1)may be defined. The discrete time instances, where changes of referencelevels may occur and gain decisions may be taken, may be determined byT1. This value may be used indirectly to set time intervals such as T1/₂and T1/₄, or other intervals, which may be sample time intervals formeasuring pixel saturation. A maximum exposure time may also be defined(T_(M)). Typically, T1 and T_(M) are both programmable. Typically, T_(M)does not impact on the calculations, other than to set a maximalexposure time at which point exposure may be stopped, regardless ofwhether exposure saturation thresholds have been crossed. T1, on theother hand, may be used as the target exposure time. For example, T1 mayrefer to the expected exposure for adequate or complete saturation etc.At intervals T1/₄ and T1/₂, for example, the current system or methodaccording to some embodiments sets the saturation threshold levelsexpected to be crossed before T1.

Typically, the device 30 transmits image information in discreteportions. Each portion typically corresponds to an image or frame. Othertransmission methods are possible. For example, the device 30 maycapture an image once every half second, and, after capturing such animage, transmit the image to the receiving antenna. Other capture ratesare possible. Typically, the image data recorded and transmitted isdigital color image data, although in alternate embodiments other imageformats (e.g., black and white image data) may be used. In oneembodiment, each frame of image data includes 256 rows of 256 pixelseach, each pixel including data for color and brightness, according toknown methods. For example, in each pixel, color may be represented by amosaic of four sub-pixels, each sub-pixel corresponding to primariessuch as red, green, or blue (where one primary is represented twice).The brightness of the overall pixel may be recorded by, for example, aone byte (i.e., 0-255) brightness value. Other data formats may be used.

According to one embodiment, a reliable exposure measurement may requireinclusion of every n^(th) (e.g., 4^(th)) pixel (which may be, forexample, every second RED pixel, since there are typically more redpixels according to one embodiment, in every m lines (e.g. 10 out of 256lines, in a typical approximately 66,000 pixel frame (e.g., 256×256pixels)). This is equivalent to approximately 640 pixels in a typicalframe. In one embodiment a reliable exposure measurement may requireapproximately 1.5% (for example, 11 pixels out of 640) of the selectedpixels to be saturated in order to pass a saturation threshold,according to which gain decisions may be taken. Other frame sizes,percentages, and sample rates may be used, as appropriate. For example,9, 11, 15, 24, and any other number of pixels can be used per frame orper sampled subset to determine a saturation threshold. Other individualpixels, e.g., non-red pixels, may be sampled, and sampling need not bebased on color.

According to one embodiment, exposure time may be determined in, forexample, 8 steps, from 5 ms to 40 ms. Other numbers of steps andintervals may be used, and intervals need not be used—e.g., exposuretime may be determined on a continual basis. Furthermore, T1 may bedigitally programmable, for example, in 8 steps with a, for example,logarithmic scale from, for example, 1 ms to 100 ms. According to oneembodiment, the accuracy of detection levels may be defined, forexample, as less than 5%. Other accuracy levels may be used, asappropriate.

The measurement of the exposure, according to one embodiment, may beperformed on a subset of the pixels in the sensor array or imager. Adetermination as to whether a gain exposure change may be in order maybe based on, for example, a percentage of the selected pixels that aresaturated with light, relative to, for example, a saturation thresholdfor one or more pixels. The gain or other parameter setting decision maybe based on one or more discrete time intervals within which the outputsfrom a nominal number of the selected number of pixels (e.g., 11according to one embodiment) have reached a certain saturation level(e.g., reference level), or threshold. In this way, the exposure maycontinue until it is determined that the pixel output from the nominalnumber of pixels has reached a new saturation level, at which time thegain (or other parameter) level may be changed. According to oneembodiment, exposure may continue until, for example, full saturation ormaximum exposure time (T_(m)) is reached. It should be noted thatdetermination of full saturation may differ according to the variousgain (or other parameter) levels. For example, expected saturation atgain 1 may be V1, and expected saturation at gain 4 may be V1/₄. Thereference voltage (Vref), or threshold voltage, for determining whethergain and/or exposure need to be changed may be defined at any discreettime interval, relative to the proportion of time to T1. For example,Vref may be equal to Vfs/₄ at time T1/₄, when the light reflection mayinitially be measured. Similarly, Vref may be equal to Vfs/₂ at timeT1/₂, when the light reflection may subsequently be measured.

According to some embodiments of the present invention, an aggregategray scale for the selected pixels (640 in the current example, etc.)may be measured at one or more intervals. The average may be compared toa saturation threshold, and gain decisions that are taken may be relatedto the average gray scale measurement and the relevant saturationthreshold.

According to some embodiments of the present invention, the signalsaturation level may be defined as Voltage Saturation, or Vsat, whichrepresents a low pixel voltage, and may be defined as the pixelsaturation referred to ground. Pixel reset level may be defined as Vrst,which represents the highest pixel voltage. Finally, Vfs may be definedas the A/D full-scale voltage, which represents the actual saturation orvoltage level at a determined interval. Accordingly, Vsat=Vrst−Vsat=Vfs.In other words, the pixel signal level may be defined as the differencebetween the pixel reset level and the instantaneous pixel voltage, andmay be a positive voltage, increasing from 0, during exposure. This“delta-voltage” may be compared with the comparator reference voltage,such as, for example Vfs/₄, at T1/₄.

Reference is now made to FIG. 15, which illustrates, according oneembodiment of the present invention, gain setting decisions that may bemade, during an exposure period, as a function of pixel output (lightsaturation) vs. time. As can be seen in FIG. 15, for example, three gainsettings that may be used are gain 1, gain 2 and gain 4. The thickerlines 161, 163 and 165 represent the gain limits or thresholds. Thesettings of FIG. 15 are provided as an example only, and are not meantto be limiting. Other numbers of gain settings may be used.

At an interval such as T1/₄, the saturation threshold (level) forselected pixels may be measured. The saturation threshold may be at,below, or above an expected threshold 161. In case “b”, where the pixeloutput (average saturation of the selected pixels) is above the expectedthreshold 161, it is to be expected that full exposure will be completedwithin, for example, T1. Therefore no increase in gain, or sensitivity,is necessary, and exposure continues until full saturation (Vfs) isreached, at which time exposure is stopped.

In case “c”, where the pixel output is below the expected threshold 161,but above a middle threshold 163, it is to be expected that exposurewill not be completed within T1. Even though saturation may eventuallybe attained, the exposure will inevitably be longer than T1, which maycause a blurring effect. Therefore the imager may be commanded toincrease the gain level from gain 1 to gain 2. Accordingly, thesaturation threshold may be decreased by, for example, half, to Vfs/₂,which together with, for example, gain 2 amplification may provide fullimage exposure. The exposure may then continue until saturation levelVfs/₂ is reached, at which time exposure may be stopped. Vfs/₂represents full saturation in this case since gain level 2 was applied.Other gain levels may be used.

In case “d” where the pixel output is below the middle threshold 163,but above a lower threshold 165, it may be expected that exposure willnot be completed within T1. Therefore exposure continues and, inaddition, a more significant increase in gain level may be necessary.The imager may be commanded to increase the gain level to gain 4, forexample, and exposure may subsequently continue until correspondingsaturation level Vfs/₄ is reached, at which time exposure may bestopped.

In case “a” where the pixel output is below the lower threshold 164, itmay be expected that exposure will not be completed within T1, or evenwithin T_(M). Therefore an increase in gain level to gain 4, forexample, may be necessary, and exposure may continue until T1. In thisexample, since full exposure has not been achieved even at T1, exposuremay continue until T_(M), to increase the possibility of sufficientexposure. T_(M) may be the maximum exposure time for all cases.

As can been in the above example, and in relation to FIG. 15, fullexposure has been achieved in 3 out of the 4 frames, and exposure timehas likewise been reduced in 3 out of the four frames, possibly leadingto significant increases in exposure clarity and decreases in usage ofpower resources. In alternate embodiments, other levels of alteration orimprovement may result.

Additional measurement instances may also be provided, such as, forexample, T1/2 and T1/₃ etc. The pixel exposures in the above scenariosmay likewise be measured at this second interval, to establish whetherfurther gain level changes are necessary.

In one embodiment, a first scan may be utilized to search for “whitespots”, or “hot spots”, which are problematic (poorly ornon-functioning) image receivers that may not be counted in the group ofsaturated pixels. This first scan may therefore be designed to detect,define and discard the problematic or non-functional pixels from theselected pixel group. Such a defective/non-functioning scan need not beused.

Following is an example of a non-limiting description of “pseudo-code”,which may be used to implement an embodiment of the present invention.Other embodiments of the present invention may be implemented withoutcoding, such as by using circuit design. Various embodiments of thepresent invention may be implemented using different code sequences andprogramming or logic design techniques:

Vref=Vfs/₄ Gain=4 Exposure=‘on’ White_spots=Scan result WhileExposure=‘on’   Pix_above_ref=Scan result %%%%For each scan   If(t>=Tm)    Exposure=‘off’   Else %(t<Tm)     If(Pix_above_ref-White_spots>=‘table_value’)       If (Gain=4)         If(t<T1/₄)           Vref=Vfs;           Gain=1;         Else %(t>=T1/₄)          If (t<T1/₂)             Vref=Vfs/₂;             Gain=2          Else % (t>=T1/₂)             exposure=‘off’           End%(t<T1/₂)         End %(t<T1/4)       Else %(Gain !=4)        Exposure=‘off’       End %(Gain=4)     End %(Pix_above_ref-White_spots>=‘table_value’)   End %(t<Tm) End %( WhileExposure=‘on’)

It is noted that such automatic gain control may result, under certainconditions in changes in the signal to noise ratio (S/N) of the imagerin some cases. For example, increasing the pixel amplifier gain in CMOSpixel array imagers may result in higher S/N ratios, while increasingthe exposure time (Tm) may increase the image “blur”.

According to some embodiments of the present invention, the abovedescribed principles and related FIG. 15 may likewise be used todescribe implementations of the present invention for other parameterssuch as illumination intensity and duty cycle etc., or in combinationwith such other parameters. For example, a method may be followed fordetermining light saturation in relation to light intensity and dutycycle etc., according to threshold values, at selected instances. Thesecalculations of saturation level may determine whether such parametersare at, below or above a given threshold. Such a determination mayenable decision making such as to change light illumination, duty cycleetc. to reflect the exposure needs of the relevant frames.

For example, the controller or any other element may measure lightsaturation levels in one or more light measuring elements, and inresponse to resulting measurements, may simultaneously at least one ofcontrol illumination duration, illumination intensity and/or image gainlevel. According to one embodiment of the present invention,illumination (exposure) may be increased eight-fold, for example, usingthe following or other possibilities:

a. duration×2, intensity×2, gain×2

b. duration×8, intensity×1, gain×1

c. duration×1, intensity×8, gain×1

d. duration×1, intensity×1, gain×8

e. duration×1.5, intensity×1.5, gain×3.56 etc.

Any other combinations of parameters that may be required to implementthe above or alternate illumination targets may be utilized. Additional,any other suitable parameters may be factored in to the above describedembodiments, individually or in any combination, to enable an in-vivoimaging device to provide more accurate exposure.

According to some embodiments of the present invention, a method isprovided for determining the location or position of an in vivo device,such as an in vivo imaging capsule. For example, a method is providedfor determining when an in vivo device enters into a body or enters intoa particular area of the body, etc. This determination may be used fordecision making, such as, for example, a decision whether an in vivocapsule should enter or exit an operational mode such as “fast mode”,“slow mode”, “standard mode” etc. A fast mode, for example, may enablethe imaging device to attain an increased frame rate, which may beparticularly useful for enabling rapid imaging of the esophagus afterswallowing an imaging capsule. Such rapid imaging is not required,however, when the device is in, for example, the small intestine,therefore the device may be programmed to switch to a “standard mode”after a time period following the internalization of the device. Otheroperational mode changes may be effected. Thus, for example, the imagingdevice may send compressed data in “fast mode” when traveling down theesophagus, and then operate in regular uncompressed mode at a lowerframe rate thereafter, when such a fast frame rate is not required.

In one embodiment of the invention, this may be accomplished by settinga mode, such as a fast mode, to end when a significant change has beendetermined in the environment surrounding the device. This may beaccomplished by providing an environmental monitoring tool, such as a pHindicator, temperature gauge or light level indicator etc. in, on oroutside of the imaging device to measure or otherwise determineenvironmental data. The monitoring or measurement tool may compare themeasured data with previously measured environmental data to determineenvironmental changes. When, for example, the environment of the capsulehas fallen below a certain pH level, this change in pH level mayindicate that the esophagus has been traversed and that the imagingdevice is in the stomach. Various environments or environmental changesmay be determined, depending on the measurement tools being used, suchas outside the body, inside the body, in the mouth, in the throat, inthe esophagus, in the stomach, in the small intestine etc.

In other embodiments, a mode, such as a fast mode, may be set to end afixed amount of time, e.g. five minutes, after a change is detected. Achange may be, for example, the capsule entering a dark environment suchas the mouth. For example, a controller may configure a light source toprovide a “dark frame” at determined frame intervals, such as, forexample, at 1 frame out of every 256. During a dark frame, LEDs or otherillumination sources may not be lit or may be lit for a brief instant,substantially inadequate to provide viable exposure for an image. Forexample, an in vivo imaging device may require a 25 ms exposure at afixed light intensity to adequately light an internal lumen, yetpurposefully provides an inadequate exposure of, for example, 5 ms. Thedevice may periodically process the “dark” frame, which may be analyzedto determine the presence of ambient light in the environment of thedevice. If the ambient light is above a threshold level, indicating thatthere is a substantial amount of surrounding light and that anon-substantial amount of additional light is required for the image toattain saturation, it may be assumed that the capsule has not yetentered the body and that the fast mode should continue. If the ambientlight during the dark frame is below a threshold level, indicating thatthe image requires a substantially significant amount of additionallight to attain saturation, it may be assumed that the capsule hasentered a darker environment, such as the body. In this case it may beassumed that the fast mode will no longer be necessary after apredetermined period of time, e.g., five minutes, by which time it maybe assumed that the capsule has passed through the esophagus.

According to some embodiments, the exposure of the dark frame may bemeasured in relation to a light saturation threshold for the darkframes, to determine whether and/or by how much a change in an imagergain level is required. If the dark frame requires a substantiallynon-maximal gain factor, indicating that the image has a relativelyadequate amount of light and only a relatively small gain level may berequired to reach full exposure, the device may be defined as beingoutside a body. When the dark frame requires a large or maximal gainfactor, indicating that a substantial gain is required for possiblyreaching full saturation, the device may be defined as being inside abody.

FIG. 16A depicts a series of steps of a method for determine a necessaryimage gain level during an exposure period, according to an embodimentof the present invention. In alternate embodiments, other steps, andother series of steps, may be used.

In step 500, a device, such as an in-vivo imaging device, turns on alight source.

In step 510, which may be a short sampling period, the device records(and possibly integrates) the amount of light received to at least onelight measuring element. This may be, for example, to a sensor on thedevice, or possibly to an external sensor.

In step 520, the device determines the amount of light recorded.

In step 530, if the amount of light recorded, for example, by a portionof the frame's pixels is less than a certain pre-determined value(saturation threshold).

In step 540, the image gain level may be increased, and the device maycontinue exposure (or other parameters, such as light level etc.) untilsaturation is attained 560. In this case, step 520 may be repeated at asubsequent time interval.

In step 550, where the amount of light recorded is more than a certainvalue (threshold), the image gain level may be decreased. Step 520 maybe repeated at a subsequent time. At full saturation exposure may bestopped 560.

If the amount of light recorded is substantially equivalent to adetermined saturation threshold (close in value to such a threshold suchthat the amount of light exposed can be assumed to be sufficient), acurrent gain level or light level etc., may be maintained and theexposure may be stopped when full saturation occurs, or until a maximumexposure time (T_(M)) is reached, whichever is first to occur. Theprocess may then be repeated 570 for subsequent frames.

In step 570, the above process is repeated from step 500, as, the devicemay operate across a series of imaging periods. However, the method neednot be repeated.

According to one embodiment steps 500 to 520 may be complemented orreplaced by a step wherein data from an environmental monitoring tool isanalyzed, to determine if one or more particular data measurements, suchas pH level, temperature level etc., are above, below or equal to athreshold for the particular measurement(s), or to one or more previousmeasurements. The results of such a comparison may be used to determinewhether an environmental change has occurred, and whether an appropriategain level change is in order. For example, an in vivo capsule may beadapted to carry multiple measurement tools for measuring differentaspects in the environment, including a light level indicator and a pHindicator. In frame A the capsule may have measured levels i and iiusing the two indicators listed above. In a second frame B it may bedetermined, for example, that both parameters i and ii have changedsubstantially from their measurements in frame 1. In the case where thelight level indicator reflects a darkening of the environment, and thepH indicator indicate a higher acidity, it may be determined that thecapsule has both entered the body (a darker environment) and entered thestomach (increased acidity).

FIG. 16B depicts a series of steps of a method for determining when anin vivo device has entered an area with different illumination,according to an embodiment of the present invention. In alternateembodiments, other steps, and other series of steps, may be used.

In step 600, a device, such as an in-vivo imaging device, turns on(operates) a light source.

In step 610, the device records (and possibly integrates) the amount oflight received to a light measurement element. This may be, for example,part of the imager, a sensor on the device, or possibly to an externalsensor.

In step 620, the device determines the amount of light recorded.Furthermore, the device may calculate this amount of light recorded inrelation to a saturation threshold or any other threshold, to determinea possible location of the device based on the amount of light recorded.For example, if the light recorded in a first frame is above asaturation threshold of, for example, 15 pixels out of the frame, thisindicates that saturation has easily been attained, and the device isassumed to be outside the body (in a light environment). If the lightrecorded in a second frame is below the same saturation threshold of,for example, 15 pixels out of the frame, this indicates that saturationhas not been attained, and the device is assumed to be inside the body(in a dark environment).

In step 630, a decision may be taken to change operation mode of thedevice, depending on the amount of light recorded relative to athreshold value.

In step 640, for example, if the amount of light recorded is less than acertain value (threshold), indicating that the device is located in adarker area, the device may change the mode of operation 640 to reflectthis darker environment. For example, the device may be configured tostart operating in a fast-mode for a period of 10 minutes after enteringthe body, to enable fast imaging of the esophagus area. Afterdetermining that the device has entered the body, the timer may beinitiated, so that after 10 minutes the device will change into a slowermode for the remainder of the procedure.

In step 650, for example, if the amount of light recorded is more than acertain value (threshold), indicating that the device is located in alighter area, the device may change the mode of operation 650 to reflectthis lighter environment.

FIG. 16C depicts a series of steps of a method for determining an invivo device's location, according to an embodiment of the presentinvention. In alternate embodiments, other steps, and other series ofsteps, may be used.

In step 700, a device, such as an in-vivo imaging device, operates atleast one environmental measuring device, such as, for example, a pHlevel sensor and a light detection meter.

In step 710, the device records a measurement, such as pH level forexample, received to the measurement device. This measurement device maybe, for example, a sensor on or in the device and/or an external sensor.

In step 720, the device determines the quantity and/or quality of themeasurement recorded.

In step 730, the device may determine a location in a body based on themeasurement data recorded, as compared with a threshold value orprevious measurements etc. For example, if the pH level in a first frameis above a saturation threshold of, for example 7 on the pH scale, thisindicates that the device is in a non-acidic environment, and may beassumed to be in the throat area (in an acidic-neutral environment). Ifthe pH level recorded in a second frame is below the same threshold of,for example, 7 on the pH scale, this indicates that the device is in amore acidic environment, and may be assumed to be in the stomach orintestine area, depending on the pH level recorded.

In step 740, a decision may be taken to change operation mode of thedevice, depending on the measurement data recorded relative to athreshold or alternative value.

In step 750, for example, if the amount of measurement data quantityand/or quality is more or less than a certain value (threshold or othervalue), indicating (or verifying) that the device is located in adifferent area, the device may change the mode of operation to reflectthis new environment.

The results of the above processes may be used to determine whetherenvironmental conditions have substantially changed, based on resultsfrom various optional monitoring and/or measuring tools. The changerequired to be defined as “substantial” or “significant” may bedetermined for each case or by manufacture.

Typically, the various embodiments discussed herein may be implementedin a device such as device 30 (FIG. 2); however, such embodiments may beimplemented in a variety of imaging or sensing devices, varying instructure. The various functions and processes described above,including but not limited to processing, monitoring, measuring,analyzing, defining, tracking, comparing, computing, commanding,stopping exposure, increasing gain, decreasing gain, increasingexposure, decreasing exposure, changing mode etc. may be executed by,for example, a processor unit (e.g., 36 in FIG. 2). These functions orprocesses may additionally and/or alternatively be implemented by theprocessor unit 36 alone, by alternative units, such as illuminationcontrol unit 40, telemetry unit 34, light sensing units 42, imaging unit32 etc., or any combination of units. The methods and processesdescribed may also be embodied in other sensing devices having otherstructures and other components. Alternately, part or all of theanalysis or control involved in the various methods presented may beperformed by an external workstation or processing unit.

It will be appreciated by those skilled in the art that while theinvention has been described with respect to a limited number ofembodiments, many variations, modifications, combinations and otherapplications of the invention may be made which are within the scope andspirit of the invention.

The invention claimed is:
 1. A method of operating a swallowable in-vivoimaging capsule, the capsule comprising a light source and an imager,the method comprising across a plurality of imaging periods, eachimaging period including a sampling period and a subsequent illuminationperiod: during the sampling period, operating the light source toprovide illumination and recording an amount of light reflected to theimager; calculating, based on the amount of light recorded, a calculatedintensity or duration of the illumination to be provided during theillumination period; and during the illumination period, operating thelight source to provide illumination at said calculated intensity orduration.
 2. The method of claim 1 wherein each imaging period comprisesa dark period during which pixel information is read.
 3. The method ofclaim 1 comprising scanning the imager to acquire an image.
 4. Themethod of claim 1 wherein the recording is based on a signal from theimager.
 5. A swallowable in-vivo imaging capsule, comprising: a lightsource; an imager; and a controller, the controller to, across aplurality of imaging periods, each imaging period including a samplingperiod a subsequent illumination period: during the sampling period,operate the light source to provide illumination and record an amount oflight reflected to the imager; calculate, based on the amount of lightrecorded, a calculated intensity or duration of the illumination to beprovided during the illumination period; and during the illuminationperiod, cause the light source to provide illumination at saidcalculated intensity or duration.
 6. The capsule of claim 5 wherein eachimaging period comprises a dark period during which pixel information isread.
 7. The capsule of claim 5 wherein the controller is to scan theimager to acquire an image.
 8. The capsule of claim 5 wherein arecording of the light by the controller is based on a signal from theimager.
 9. A method of operating a swallowable in-vivo imaging capsule,the swallowable in-vivo imaging capsule comprising a plurality of LEDsan imager, the method comprising across a plurality of imaging periods,each imaging period including an illumination period and a samplingperiod: during a sampling period operating the plurality of LEDs andrecording an amount of light reflected to the swallowable in-vivoimaging capsule; calculating, based on the amount of reflected light, acalculated intensity or duration of the illumination to be providedduring an illumination period; and during the illumination period,providing illumination using the plurality of LEDs at said calculatedintensity or duration.
 10. The method of claim 9 wherein each imagingperiod comprises a dark period during which pixel information is read.11. The method of claim 9 comprising scanning the imager to acquire animage.
 12. The method of claim 9 wherein the recording is based on asignal from the imager.